The present invention relates to a device for detecting an object which is present on a plane, in particular, an obstacle present on the surface of the road of travel of a moving body, such as an unmanned dump truck, or the like.
At present, moving bodies, such as unmanned dump trucks, or the like, cannot be regarded as having adequate capability for detecting obstacles present on their road of travel. This is because a method is still used whereby obstacles in the forward direction of travel of a moving body are detected using ultrasonic sensors, laser readers, and millimeter-wave sensors, and when such a method is adopted, problems arise in that the range of detection of obstacles is restricted by the narrowness of field of vision, and only a small amount of information is obtained from the sensors.
On this point, if a method is adopted whereby images are gathered in the forward direction of the moving body and obstacles are detected by processing the images thus taken, merits are obtained in that the amount of information gathered is large, the field of view is broad and hence obstacles can be detected across a wide range.
There exist obstacle detectors based on image processing of this kind which are devised such that, by using a stereo camera (at least two cameras) and matching the pixels in the images captured by these two cameras on the basis of a pattern matching principle, a distance image comprising two-dimensional co-ordinate position information and information on the distance from a reference point on the moving body is generated, and the distance from the reference point on the moving body to the obstacle can be detected using this distance image.
A distance image of this kind may be a distance image having sparse pixels which comprises distance information for the outline regions of a detected object only, or a distance image having dense pixels which shows pixel information for the surface regions of the detected object as well as the outline regions thereof, and by differential processing of the dense distance image, the outline regions of the detected object only are displayed and a distance image having sparse pixels is obtained.
As a distance measuring device of this kind, it is possible to use the device disclosed in Japanese Patent Application Hei.7-200999 (title of the invention: xe2x80x9cObject distance measuring device and shape measuring devicexe2x80x9d), which relates to the present applicants.
However, since the distance images used are viewed from the camera co-ordinates of a stereo camera mounted on the moving body, in cases where the moving body is inclined at a steep gradient during its travel or cases where the road of travel becomes a slope, the road itself may be photographed in a high position in the distance image, and in many instances, it becomes difficult to distinguish whether an object displayed in the distance image is the road itself or an object that is present on the road.
Moreover, the distance image may contain many mismatched points, and if the distance of an obstacle is detected by processing the image as it is, then the distance to the obstacle may be misdetected.
Furthermore, since the position of an obstacle with respect to a moving body changes progressively, it is desirable that image processing for detecting an obstacle is carried in a short time, in order that the position of the obstacle is displayed, and the like, in real time.
The present invention was devised with the foregoing in view, a first object thereof being to make it possible readily to discriminate between a road of travel and an obstacle in a distance image, even in cases where such discrimination is difficult, for instance, when a moving body is inclined during travel or when the road of travel thereof forms a slope.
It is a second object of the invention to make it possible to detect an obstacle accurately, even if is supposed that distance images contain many mismatched points.
It is a third object of the invention to compress processing time for detecting obstacles, in such a manner that the position of an obstacle can be detected, displayed, and the like, in real time.
The present invention can be applied not only to detecting obstacles on the road of travel of a moving body, but also to detecting all types of objects present on a plane.
Therefore, in order to achieve the first object described above, a first aspect of the present invention is an object detector comprising distance image generating means for measuring a distance from a reference position to an object to be detected present on a plane and generating a distance image of the plane and the object to be detected, and detecting means for detecting the object on the plane by using the distance image generated by the distance image generating means, characterized in that the object detector comprises: three-dimensional distribution generating means for calculating three-dimensional co-ordinate position data in a three-dimensional co-ordinates system, for each pixel of the distance image, on the basis of two-dimensional co-ordinate position data for each pixel of the distance image and distance data from the reference position for each of the pixels, and generating a three-dimensional distribution of pixels corresponding to the plane and the object to be detected; and detecting means for calculating a group of pixels corresponding to the plane on the basis of the three-dimensional distribution generated by the three-dimensional distribution generating means, and detecting groups of pixels having a prescribed height or more with reference to this calculated plane as the object to be detected.
A representative example of an object to be detected on the aforementioned plane is an obstacle on the surface of a road on which a moving body is travelling.
In other words, according to the first aspect of the invention, three-dimensional co-ordinate position data in a three-dimensional co-ordinate system is calculated for each pixel of a distance image, on the basis of two-dimensional co-ordinate position data for each pixel of the distance image and distance data from a reference position for each pixel, in such a manner that a three-dimensional distribution of pixels corresponding to a plane and the object to be detected is generated, and therefore the object to be detected corresponding to groups of pixels having a prescribed height or more with reference to the plane can be distinguished readily from the group of pixels corresponding to the plane, on the basis of this three-dimensional distribution. In other words, it is possible readily to distinguish between a road and obstacles in an image, simply by means of a distance image, even in cases where such a distinction is difficult to make, for instance, when the moving body is inclined during travel or when the path of travel is on a slope.
In order to achieve the second object described above, a second aspect of the present invention is the first aspect of the invention, characterized in that the distance image generating means generates the distance image by matching corresponding points of two images captured by two cameras, and when the three-dimensional distribution is generated by the three-dimensional generating means, pixels in the distance image obtained previously from the distance image generating means which show a matching error for the matching operation that is equal to or above a prescribed threshold value are regarded to be mismatched points and are removed.
In this way, according to the second aspect of the invention, even in cases where the distance image contains a large number of mismatched points, obstacles can be detected on the basis of the generated three-dimensional distribution after previously removing the mismatched points, and therefore misdetection of obstacles can be prevented and obstacles can be detected with good accuracy.
In order to achieve the third object described above, a third aspect of the present invention is the first aspect of the invention, characterized in that the distance image generating means generates the distance image of the object to be detected including a surface of the object, and when the three-dimensional distribution is generated by the three-dimensional distribution generating means, an edge distance image of the object to be detected representing only the edges of the object is obtained by differential processing of the distance image previously obtained from the distance image generating means, and the three-dimensional distribution is generated by using the edge distance image.
Alternatively, in the first aspect of the invention, the distance image generating means generates an edge distance image of the object to be detected representing only edges of the object, and the three-dimensional distribution generating means generates the three-dimensional distribution by using the edge distance image.
In this way, according to the third aspect of the invention, a dense distance image of the object to be detected including the surface of the object is converted to an edge distance image of the object representing only the edges of the object to be detected, or alternatively, an edge distance image of the object to be detected is generated by the distance image generating means, whereupon image processing for detecting obstacles is carried out on the basis of this edge distance image, and therefore it is possible broadly to reduce the number of pixels searched when processing the image, thereby significantly compressing the search time. In other words, the processing time for detecting obstacles is compressed, and hence processing, such as detecting and displaying obstacles, can be carried out in real time.